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This study tested whether the word association employed by individual students 
could be used to predict their performance in a particular course. It was designed td 
explore a new method for describing the various models used in thinking and to 
determine whether this approach would yield results that were consonant with curren 
cognitive theory. At the beginning and end of an introductory course in Socia 
Psychology, college freshmen and their instructor filled out association matrices for 
20 words that were central to the course. Using 1 word as a subject, the students 
assigned associability scale values to all possible pairs of words on a 7-point scalCi 
indicating to what degree each of the other words would "fit in* with (top of scale) or 
change (bottom of scale) the subject. At the end of the term the association matrices 
were analyzed by a new, non-parametric method of factor analysis, the Matri) 
Optimizing System (Mopsy). Two major hypotheses of the study were confirmed with 
levels of correlations in the .3 to .4 range: students whose associative matrices yieic 
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Introduction 



During the last fifteen years there has been a considerable revival 
of Interest in word association* This revival has been built on four 
recent developments: 

(1) The Increase in cognitive studies with the realization 
that some sort of complex mediation process was necessary to 
account for human behavior , and the recognition of the central 
position of language in that process* 

(2) The remarkable development of linguistics with its 
emphasis on language structure at all levels, the present 
plausibility of machine translation, and the consequent focus 
on semantics as a linguistic bottleneck which has yet to yield 
its structural secrets* 

(3) A gradual maturing of interest in idiographic studies, 
as nomothetic approaches are mined out, one by one, and a 
significant residue remains* 

(4) The rapidly advancing state of computing hardware 
which makes feasible complete idiographic analyses and the 
processing of the massive data typically generated in verbal 
studies; and software, the variants of factor analysis and 
multi-dimensional scaling which search for the latent structures 
that linguistics tells us to expect* 

My concern in this project has been with the derivation of associa- 
tive structures, rather than with the basic mechanism* The references 
have been chosen with this concern In mind* I'Jhlle they are not exhaus- 
tive, they do represent all the major contributing lines of thought. 

The first significant chain is found in clinical psychology. Jung*s 
Studies in Word Association (20) Introduced the idea that a systematic 
study of free associations offered a glimpse of a patient's Internal 
life* The emphasis was primarily on the aberrations or blockages in 
association, which were thought to indicate specific emotional problems* 
Schafer (43) asked what was the total set of processes lying behind 
the word association test, pointing up several which had not been em- 
phasized by the earlier clinical interest In "complexes". Laffal (22), 
using techniques associated with the study of verbal learning, supported 
the original hypotheses of association testing by showing that disturbing 
words took slightly longer to learn than neutral words in a paired 
association task, and that they produced more variety In free associa- 
tions* Moss (30) was an early user of the semantic differential who 
tentatively defined successful therapy as the approximation of affective 
meanings at the conscious level with those of the unconscious, reached 
experimentally via hypnosis* Brody (8) continued the process of bringing 
the studies of normal and disturbed language together by pointing out 
that subjects with high anxiety present essentially exaggerated forms 
of the normal response to word association tests. That Is to say, they 
show excessive commonality to words which normally elicit it, and ex- 
cessive variety to words generally eliciting low commonality. Laffal 
(25) suggested that patterns of response to association tests grow 
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largely out of general patterns of superordinate categories, and offered 
a semantic classification scheme for application to clinical content 
analysis. The sequence in all these clinical studies seems to be from 
a study of the particular, aberrant response to a study of the pattern 
of all responses, normal and abnormal, on the grounds that both are neces- 
sary for the understanding of either. 

A number of recent approaches to content analysis show the opera- 
tion of a similar trend. Straight frequency studies of special content 
categories have been partially superseded by variants of continguency 
analysis. Pool (37) gives an idea of the range of this work, including 
an experimental demonstration by Osgood of the validity of the assumption 
that co-occurrence is a measure of association in the source. The new 
approaches to content analysis find application in two papers by Laffal 
(23, 24) in the clinical field. One is a contingency analysis, using 
Laffal* s broad category list, of "the contextual associates of sun and 
God" in the autobiography of a psychotic, for the purpose of checking 
rival psychodynamic hypotheses about him. The other paper examines 
changes in category profiles of a schizophrenic patient during therapy. 

It is interesting to note that continguency analysis, before its recent 
vogue, was applied to the study of a single personality by Baldwin (2), 
with an eye to identifying a personal structure. Allport (1), editing 
the same set of letters that Baldwin worked on, comments on the general 
agreement between subjective analyses and statistical-structural analyses 
of the case, and at the same time renews his plea for idiographic analysis. 

A somewhat different tack has been taken by the students of verbal 
learning. Their efforts have aimed at finding the principles of associa- 
tion operating across subjects. The various papers by Bousfield et al 
(5,6,7) focus on the phenomenon of clustering of associated words in 
the recall of randomly ordered word lists containing mixtures of cate- 
gories - a further validation, if you like, of the content analyst* s 
assumptions, Jenkins (18) offers a variety of experimental work on 
ttie causes and conditions of verbal learning and word association, notably 
(for our purposes) including a detailed study of two normal college 
students with widely differing associative habits and apparently with 
analogously contrasting personalities. Ervin (12), in a broad discussion 
of paradigmatic vs. syntagma tic association, shows how the factor of 
expectancy in listening would provide an explanation for paradigmatic 
associations. In general, the papers cited here seem to be moving, via 
massive experimentation, toward more and more complex mediating systems 
for verbal behavior. 

An early set of papers should be mentioned in this context. Razran 
(39,40) and Riess (41,42) demonstrated that classical conditioning with 
GSR and salivation as UCRs and words as CS would yield stimulus generaliza- 
tiotx to associates of the conditioned word, that the amount of general- 
ization to certain classes of words matched the established results of 
free association tests, including the known changes with age, and that 
the conditioning could take place without awareness. Even these early 
results offered the possibility of a kind of semantic distance mapping. 
Notice that the conditioning experiments did not require a stimulus- 
response theory. On the contrary, conditioning was only a measuring 
device operating upon an independently established associative system. 
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Combining the verbal learning theorists* interest in traditional 
association data with the naive directness of the conditioning experi- 
ments are a group of studies 1 would put under the label "associative 
similarity", Flavell (13) and Flavell and Stedman (14) discussed the 
relation of the semantic differeritial and co-occurrence to direct judg- 
ments of similarity of meaning, arguing the underlying complexity of 
those judgments, but showing that they could not only be made reliably 
but were remarkably stable from grammar school through college* Deese 
(9) introduced the concept of associative overlap and indicated that 
preliminary data from this measure were related to such things as 
Rorschach response patterns and value profiles - areas of high value 
showing higher association levels and more patterning. Marshall and 
Coffer (29) reviewed ten different measures of association, commenting 
on their particular values and shortcomings. They note that these 
indices had been shown to predict, variously, "free recall of verbal 
lists, mediated transfer between verbal lists, perceptual word- 
recognition memory, and conditioned generalization". My own effort (17) 
at an associative index strives to use a more "natural" measurement 
situation and at the same time to recognize the logical directionality 
of association in complex structures. The art! .le by Johnson (19) is 
an application of Deese *s associative overlap measure to an educational 
setting. As such, it is of major interest to me, and for that reason 
1 shall postpone its discussion until later. Deese (10) summed up 
several years of his and others* work on associations, with extensive 
application of factor analysis to bring out the associative structure. 

It is important to note that Deese now presumes - and with good reason - 
to proclaim in his title that these associative structures are not only 
in the language but in thought as well. This brief comment cannot do 
justice to his book, but one point in it is worthy of special mention 
here: 1 suspect that his comparative study of factorial structures in 

adjectives vs. nouns, demonstrating the continuous and non- continuous 
character, respectively, of their semantic spaces, will prove to be of 
very great importance. 

The sequence of semantic differential (SD) research reports begins 
with Osgood and luria (33). This blind analysis of the "three faces of 
Eve", produced as an early byproduct of more theoretically oriented 
research, represents a significant success in applying relatively 
simple semantic solutions to complex clinical problems. The SD is 
basically a questionnaire presentation of a number of bipolar adjective 
scales upon which the respondent is asked to rate several concepts. It 
is the nature cf such an elegantly simple design, of course, that vari- 
eties - including nonverbal - of concepts and scales may be used. To 
the respondent's answers some form of factor analysis is applied to 
yield, variously, clustering of scales across subjects, concepts across 
subjects, or concepts or scales within one subject. This last result, 
intra-eubject structure, is an especially significant one, lacking in 
Deese *s associative overlap method, but meeting the need for idiographic 
analysis. As in the Osgood and Luria paper mentioned above, this poten- 
tial was used by Lazowick (26) and Luria (28) to compare characterizations 
of self and parents in anxious and neurotic groups and their controls. 

In the study of attitudes, Prothro and Keehn (38) applied the SD to the 
description of national stereotypes. 
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The basic text for users is, of course, Osgood, Suci and lanncnbaun 
(34), Starting with a basic theory of mediation processes in verbal 
behavior, they described the development and selection process for the 
SD and showed a variety of applications to clinical, attitude description, 
and attitude change problems. Any of these could be looked at either 
from the individual or the group perspective. The 1960 article by Moss 
(31) reviewed the spate of SD research, noted its clinical, anthropo- 
logical, and social psychological applications, and pointed out that its 
validity has been clearly established, although the interval character 
of its scaling remained questionable. A series of papers (35, 45, 46, 48) 
demonstrated the cross-cultural constancy of the basic affective meaning 
structure, but expected variations in adjective scale makeup and in the 
scoring of particular concepts. At the same time, SD results, according 
to Osgood (35), showed a disappointing lack of correlation with IQ and sex. 

A final, clear ly-definable strand of influence in this area comes 
from linguistics, especially its anthropological phase. The linguist's 
tendency is to look for complete, airtight analyses as opposed to 
statistical ones. In semantics this is a questionable ambition, but the 
study of kinship terms is a special case. Goodenough (15), Lounsbury (27) 
and Wallace and Atkins (49) discussed particular kinship systems in detail, 
defining the semantic components necessary to generate the systems and 
morphemic equipment used to convey them. Goodenough argues that gaps in 
the componential structure are likely loci of linguistic and/or conceptual 
change. In another article, Wallace conjectures, on the basis of kinship 
data, that 64 terms (the equivalent of six binary distinctions) is a 
functional maximum for folk terminologies, as distinct from expert or 
professional vocabularies. He supports the conjecture with numerous 
examples from anthropology and everyday life. 

The best summary I know of for the present state of the semantic art 
in linguistics is Weinreich (52). In this overview Weinreich attempted 
to present the range of possibilities of semantic components and their 
verbal vehicles in all languages. The variety is large, but not so large 
as to defy analysis. 

I have saved for last a motley series of references that might have 
been parcelled out among the other classifications, but which share the 
characteristic of being especially close to my intentions because of 
their strong emphasis on associative structure. Ohman (32) reviewed a 
whole series of European linguistic studies dating from the 1930s which 
she classes as examples of field theory. The name Jost Trier was particu- 
larly associated with the examination of terminologies or of clusters of 
near synonyms with an eye to describing their network of semantic rela- 
tions and, in turn, the sociology and cognition they reflected. These 
studies were exhaustive rather than statistical in conception, and aimed 
at presenting group usage, but there seems no reason in principle why 
the same approach could not be applied to individuals. Indeed, much 
idiographic analysis by clinicians in the line of duty must be very close 
to this form of analysis. The previously mentioned study by Baldwin may 
be taken as a partial instance. 

Kelly (21) built a whole behavior and personality theory on the cog- 
process, which he argued was made up of binary constructs involving 
judgments of similarity and contrast. The comparison with both the bipolar 
scales of the SD and the recurrent theme of contrast in linguistics is 
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extremely suggestive. In the Role Construct Repertory Test, Kelly 
provided a means for locating and factor analyzing the cognitive dis- 
tinctions which anyone makes among other persons and himself, Kelly 
argued that understanding these dimensions, most but not all of which 
are verbally mediated, was tantamount to understanding the person. 

Granting this, he said, the concept of motivation becomes unnecessary, 
Itogg, Informati on Processing . by Schroder, Driver, and Streufert 
(44) is closely related to Kelly*s work, but shifts from his clinical 
emphasis to a small-group, problem-solving focus. These authors take 
personal complexity as their most important variable, showing how this 
complexity varies as the person interacts with different secto*. s of the 
environment under varying conditions. They define complexity as having 
two components: differentiation, the making of distinctions among stimuli, 
using one or several dimensions of classification; and integration, the 
process of combining these dimensional judgments, comparing them, and 
generating complex relationships among them, 

Boulding (4), it seems to me, echoed Kelly*s position when he 
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U^stlQg effects of the physics course in question, in terms of the 
semantic spaces (or cognitive maps or Images) of the students. 

The final reference in this integrative sequence is Be 1th (3). 

Belth has asserted that the discipline of education is properly defined 
as the study of the various models used in thinking. As such, it seems 
to me, it is identical with Bouldin^'s eiconics. Belth has, I believe, 
put his finger on an essential point. If we are to conduct effectively 
the process of adding to people's knowledge and influencing their 
attitudes, we must be able to state where they are, cognitively, where 
we would like them to be, and therefore the paths connecting those 
points. It is quite probably that the study, through various approaches, 
of individual models of sections of the worlfd is the necessary basis 
for the discipline of education. 

The work reported below is an effort on my part to explore a method 
for describing such individual models in an educational setting, and to 
determine whether the approach would yield results consonant with current 
cognitive theory. 

More specifically, I hypothesize that word association data for a 
single individual on a sample of words within a particular subject area 
can be analyzed in such a way as to yield useful indication of the 
person's cognitive organization of that area, as to both content and 
formal properties, such as complexity* Still more specifically, I 
hypothesize that the larger the number of dimensions extractable from 
a person's association matrix in a topic area, the more successfully 
he will deal with that area, and in particular the higher will be his 
grade in a relevant course. I hypothesize also, with some trepidation, 
that the agreement of a student's cognitive structure with that of the 
instructor will also increase the student's grade. 

Around the major hypotheses revolve several derivative ones: that 
specificity of matching to the instructor's dimensions rather than just 
sharing a cognitive space, will be more significant; that changes in 
level of organization or in various matchings will be significantly 
related to performance level. 



Method 



Subjects were 67 students, 26 men and 41 women, in a college 
freshman course called ’’Introduction to Social Psychology" At the be- 
ginning and again at the end of the course the students and the instructor 
(myself) filled out association matrices for twenty terms central to the 
course. The students were assured that their results would not affect 
their course grade, since the analysis was not to be performed until 
after the end of the term. The twenty terms were: attitude, change, 

communication, conflict, culture, follower, group, individual, influence, 
interaction, language, leadership, learning, perception, prejudice, public 
opinion, role, socialization, society, stratification. The terms were 
listed down the side and across the top of a 20 x 20 grid. The students 
were asked to imagine that they were engaged in a conversation or reading 
a book, and that the row-term was the current subject. They were asked, 
for each of the row-terms, to state by means of a 7-point scale (-3 to +3) 
to what degree each of the other terms would "fit right in" (+3) or be 
"a complete change of subject" (-3). The scale was anchored only at the 
ends. 1 made it clear that the judgments were to be personal (not "correct" 
or "incorrect"), and that the matrix was not required to be symmetric - 
that is, that the A-B transition might be different from the B-A transition. 

Of the 67 students, six completed the initial form but dropped the 
course and could not be included in the final analysis, though their 
results are included in some of the purely descriptive summaries. Only 
39 students returned the 2nd form, and one of those had done only the 
second form. N, therefore, is 66 for descriptive statements about the 
independent variables from the first testing, 60 for correlational st'ite- 
ments from first testing, 39 for correlational statements from second 
testing and 38 for comparisons of first and second testing. All findings 
are given in terms of ordinal scale statistics. 

The association matrices were analyzed individually by Mopsy (the 
Matrix Optimizing SYstem) , a form of nonparametric factor analysis. 

(Mopsy is discussed in detail, with sample analyses, in Appendix A. Since 
I consider Mopsy to be a major result of this project, I hope the reader 
will give more than usual attention to that appendix.) Mopsy' s output 
is a series of rank orders which may be treated as orthogonal dimensions of 
a factor space. Output of new dimensions continues until the matrix is 
exhausted or until a dimension sufficiently violates the presumption of 
orthogonality. (In other words, more dimensions are added as long as 
there is more information in the matrix and as long as that information 
is sufficiently non-redundant.) Mopsy is programmed in the IBM Symbolic 
Programming System for the 1620 computer, with on-line printer, 40,000 
decimal digits of core storage, and disk storage. 

The following items of information were recorded for each student 
whose data was complete (maximum subsets for others): course grade, average 
association scale values for both matrices, the match between the 
student's before- and after structures, and between those and the instruc- 
tor's before- and after structures; for each of these matchings, a 
specificity ratio; and for all the above measures, where possible, the 
change from the first to the second testing. 
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Some of the measures call for explanation. To matcli two structures » 
the student's with his own, or with the instructor's, I first correlated 
aH the dimensions of one with all the dimensions of the other. For the 
specific match, I selected the set of correlations v/hich would yield the 
highest possible total sum of squares, with the condition that no row or 
column could be represented more than once, and the set could therefore 
be only as large as the number of dimensions in the smaller structure. 

This allowed for the fact that dimensions in the two structures which other- 
wise corresponded very well might be extracted in a different order. The 
root mean square of the selected correlations was taken as a measure of the 
specific matching of structures. Another figure for total matching was 
obtained by summing the squares of all the cross-correlations, dividing 
by the smaller number of dimensions and finding the square root. Specificity 
of matching was defined as the ratio of the sum of squares of specific to 
that of total matching. 



R cau lts 



This statement of results will not be exhaustive, but will attempt 
to give an impression of the lay of the land, plus the most interesting 
correlational results. 

Grades, on a 4-point scale, varied around a median of 2,58, with an 
interquartile range of 2.32 to 2.895, and a total range of 1.09 to 3.84. 
Numbers of dimensions in the first testing ranged from 2 to 6, interquartile 
(IQ) range 4 to 5, and median 4, In the second testing the range was 1 
to 6, IQ range 3 to 4, median 4. 

Reliability measurement was a problem. The first and second testing, 
although they were separated by experience that was expected to change 
the performance, offer our only check. The number of dimensions, first 
vsn second testing, correlates ,33: a clear relation, but poor reliability 
The matching statistic ranged from .25 to .70, IQ range .34 to .47, median 
,435, The fact that the dimensions were matched by selecting the high 
values among the correlations exaggerates the level of matching. The cor- 
rect value would be close to the ,33 obtained for number of dimensions. 

Matchings of student structures to those of the instructor were, 
oddly enough, almost identical with the self-matching. The various com- 
binations yielded medians ranging from ,395 to .42, an IQ range of about 
.12, and total ranges from .16 to .79. 

Specificity measurements for the different matchings were also about 
the same, with medians ranging from ,515 to ,595 (that is, the specific 
matching of dimensions accounting for a little over half the total shared 
variance), with an IQ range of about ,16 and total ranges from ,31 to ,89, 

Changes from first to second testing, for number of dimensions, 
matchings, and specificity all varied around a zero or near-zero median,, 

Mean association values ranged from -0,16 to +2,90, IQ range +0,69 to 
+1,59, median +0,97 for first testing. Second testing yielded a range from 
+0,42 to +3.00, IQ range +1,15 to +2,41, median +1,81, a clear increase 
over the first testing, (The +3 average was produced by a single student 
who turned in her second matrix with a single, large +3 on it. See dis- 
cussion of this point below, ) 

The hypothesis that course grade and number of dimensions are posi- 
tively related was confirmed, for the first testing only, but at relatively 
low levels of correlation. The relevant figures are as follows, with 
correction for the correlation that turned up between association values 
and grades: 
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The hypothesis that matching with the instructor *s structure would 
be positively related to grades was confirmed, but mostly with respect to 
the instructor's initial structure only* The significant correlations, 
all with association value held constant, are as follows: 



Specific match, student 1st: instructor 1st, vs. grade 
Total match, student 1st: instructor 1st, vs. grade 
Specific match, student 1st: instructor 2nd, vs, grade 
Specific match, student 2nd: instructor 1st, vs. grade 
Total match, student 2nd: instructor 1st, vs. grade 
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The above correlations suffer a very slight reduction when the number of 
d im ensions is also held constant, but the picture is basically the same* 

The several measures of specificity showed no relation to course 
performance* Likewise, no measure of change produced a significant 
relation to performance. 

The next several pages give structural diagrams, together with 
relevant statistical information, to convey an idea of the Mopsy output 
and a feeling for the content of this study. Up to three dimensions are 
presented graphically, with any further orders listed below the diagram. 

When a term is not included in an order, it is because its matrix row went 

to zero at an earlier point in the analysis. The first dimension (I) is 

the vertical, the second (II) horizontal, and the third (III) depth 
(represented by the block sizes). 

The following correlations are worth noting as you examine the diagrams 
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Instructor 1st - II instructor 2nd - I -.52 
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Discussion 



The confirmation of the major hypotheses, that number of dimensions 
and agreement with instructor's structure vary with grades, suggests that 
the investigation has a piece of something, but the low correlations (,3 to <,4) 
show either that the functional relationships are inherently weak or that 
there is considerable error variance uncontrollede As might be predicted, 

I am sufficiently interested even in the weak findings to want to track 
down error variance* 

Number of dimensions . Schroder, Driver, and Streufert (1967) state 
clearly the position of current complexity theory: that a person who has 

more ways of classifying and combining chunks of information in a particular 
area is able to deal more effectively - process new information, retrieve 
old information, or solve problems - with that area* They emphasize that 
this personal complexity, while it may be linked to general personality 
characteristics, varies considerably from one topical area to another, and 
with varying conditions of reward and punishment in the environment* 

In this study, I have chosen to represent a cognitive area as a semantic 
field, given structure by the arrangement in it of chunks of information in 
the form of words* I would argue that every associative link is implicitly 
a (more or less) rudimentary theory which could be articulated by naming 
the relation underlying the linlc. ’^iJhere these links are grouped into a 
chain, cluster, or dimension we have an implicit theoretical system, with 
extended and interlocking relations and luiernal analogies* The word 
"implicit" is crucial here, however, because there may be great differences 
in the degree to which any such theoretical system is susceptible of ex- 
plicit statement and rational, discursive explanation* We may nevertheless 
expect that all the dimensions of a person's cognitive structure in an area 
will affect how he deals with that area* Differences in potential explicit- 
ness may well lie behind those curious discrepancies which, surely, every 
teacher has come upon, between a student's feeling of control over a subject 
and what he is able to articulate for formal evaluation* 

This last point raises the usual questions about the meaning of course 
grades, our only dependent variable* Even disregarding the problem of in- 
consistency in grading standards, we may well ask whether our standards do 
not overemphasi*;e verbal facility, to the disadvantage of intuitive under- 
standing and originality* In this light, correlations of *3 to .4 may well 
represent a much larger proportion of the systematic variance than of the 
total variance* But that is only speculation* 

A more mundane problem cropped up in the measurement of number of 
dimensions, typified by the student mentioned before who turned in a second 
matrix with a single, large +3 written across it* This is exemplified in 
general by the marked Increase in the mean association values from the first 
to the second testing* Of course, greater acquaintance with an area will 
provide more potential associations among all of its parts* In other words, 
more relations are put to work in classifying the information, and this is 
what we mean by increased complexity* But in my study, where only a seven- 
point scale was available to register association, students tended to push 
their judgments to the top of the scale, thus cramming them into a smaller 
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ranee. This in turn affects the factor analysis by reducing the differentia- 
tion among items and so reducing the number of distinct dimensions. Worse 
yet, the better the student, the greater his increase of associations, so 
our complexity measure gets washed out. 

In the results reported above I attempted to correct for this by means 
of partial correlation, with some good effect. But this is at best patching. 

The relation of mean association to number of dimensions is complex because 
of the end-of-scale effect. A very large mean association value necessitates 
fewer dimensions, but mid-scale values may yield few or many dimensions, 
depending on their variance. It seems quite likely that the loss of signifi- 
cance of the correlation of dimension-number with grade from first to second 
testing is a result of the increased interference from the end-of-scale effect. 

Reliability and matching. As noted above, the before-after comparison 
was a weak way of checking reliability. Since change was expected, normal 
reliability criteria could not be applied, but I did expect something on the 
order of ,6 to ,7, which was the matching level of my own first and second 
tests. Only one student reached that level, however, and the median value 
was only slightly above ,4, Still more surprising, matching with my struc- 
ture was substantially equal with self-matching. 

The end-of-scale problem may be producing interference here too. The 
correlation between mean association value on the second test and the specific 
matching reliability was -.30, Systematically larger association values on 
the second test mean fewer dimensions available for matching, and the lack 
of differentiation in the matrix may make a structure much easier to tilt i’.\to 
a new set of dimensions by means of a few small changes in associations. 

The similarity of the reliability and instructor-matching indices raises 
at least two questions. First, can it be that it is increased matching with 
the instructor that reduces the reliability? The relevant correlation, 
between change toward the instructor and reliability, is indeed negative, 
but only -.14, It is too small to solve the problem. Second, are we dealing 
with simply a general agreement as to the use of these words? The rather 
wide range of both the reliability and matching indices throws some doubt on 
this, but it is the best explanation I have to offer. The Mopsy program tends 
to play down 'general factors", but where patterns of differentiation are 
similar, it will reflect thato 

The correlation between instructor -matching and grades bears some dis- 
cussion. In view of the instructor's fairly high reliability, 't is surprising 
that matching with his second test is only related to grades in one instance, 
and not very strongly, while matching to his first test is consistently re- 
lated to grades, whether comparison is by specific or total matching, or from 
the student's first or second test. This suggests that the instructor's 
initial categorization sets the framework of the course, and students are 
more or less able to make sense of the material on that basis. Alternatively, 
we might conclude that the Instructor formed a basic impression of the student 
early in the course which was difficult to modify later on. Less cynical than 
either of those interpretations, and equally plausible, is the possibility 
that some cognitive structures are better matches than others for the "inherent" 
structure of the subject matter, and if so it seems reasonable to expect the 
instructor's structure to approach that ideal more closely than does the average 
student's. Clearly we need additional data to settle the locus of causality; 
perhaps a student's evaluation of his own understanding of the course would 
of some help, as would some criterion matrices from specialists ir, the field 
other than the instructor. 
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An additional possibility must be dealt with for the matching vs. 
grades correlation. Number of dimensions has been shown to correlate 
positively with grades. Could the greater number of dimensions, by providing 
more chances for a good match, account for the correlation? The correlation 
between dimensions and matching on the first test is +.4, and on the second 
more than +.5, but when number of dimensions is held constant the matching 
vs. grades correlation is reduced by only one or two hundredths, remaining 
statistically significant. Apparently matching and complexity affect per- 
formance independently. 

A last point on matching; I have made little mention of the distinction 
between specific and total matching, because they turned out to be nearly 
identical in distribution, and because the specificity ratio between them 
provided no significant correlations. Though the specific matching is in- 
tuitively more attractive to me, total matching is less trouble to compute. 

The structural diagrams . The diagrams, though they are among the 
long-term goals of this line of research, must be approached intuitively at 
this stageo The statistics presented above are one attempt to get a grip on 
the structures and their functional relationships to formal learning. At 
the moment we can simply ask that they 'feake sense”, and, in varying degrees, 

I think they do. I have found that a number of dimensional labels, such as 
"cognitive-social”, "psychological-sociological”, "adaptive-receptive”, 
"fixed-fluid”, "abstract-concrete”, "static-dynamic”, and "nomic-anomic” 
seem repeatedly appropriate. No doubt the reader will think of other labels. 
But in the long run the important objective is to gain an understanding of 
the student's construct and its functional relations. 



Conclusions, Implications, Recommendations 



This study offers two kinds of conclusions: 

lo Methodological. It appears that the Matrix Optimizing System 
offers a plausible alternative to parametric methods of factor analysis, 
since it makes fewer assumptions about the data, runs far more rapidly on a 
computer, and may for some purposes offer a richer or more concentrated data 
description than do the accepted methods. By making this method available 
upon request I hope to learn more about its useful applications and short- 
comings, and to learn whether a rigorous rationale is available for it, 

2, Substantive. Tito major hypotheses were confirmed insofar as 
the relations were greater than one would expect by chance, but at relatively 
low levels of correlation. It was shown that students whose associative 
matrices yield a larger number of dimensions under nonparametric factor 
analysis, tend to get higher grades in the course, as would be predicted from 
current complexity theory. Second, it was shown that students whose cognitive 
dimensions matched those of the instructor tended to get higher grades in the 
course. But the findings were static. None of the measures of change of 
structure was found to be related to course performance. 

An end-of-scale effect in the association judgments produced considerable 
interference with the results, especially the "after".measures. It is apparent 
that this will have to be corrected before relations strong enough to be useful 
can be found. 

In order to improve the method and design, I propose the following 
changes; (1) Remove the end-of-scale problem by specifying a distribution to 
which the association judgments must conform. The obvious choice here is a 
rank ordering of each row, with the highest value given to the word most 
closely associated with the head word of the row. (2) Use fewer subjects 
and get more matrices from each - perhaps five in one term. This will allow 
a closer check on reliability and throw some light on the gradual processes 
of change, (3) Add more dependent variables. Course grades are worth using, 
but they should be supplemented by such measures as instructor's judgments 
of students' originality, and students' statements of both feeling and under- 
standing about the Instructor and different books in the course, all of whose 
associative structures can be derived for comparison. 

All of this proposed follow-up of findings which accounted for only 
10-15% of the variance in the dependent variable may seem risky. But in my 
opinion the value of a systematic but rich method of describing associative 
structure and predicting its effects in an educational setting is so great 
as to justify following up anything that offers consistent and statistically 
significant results, however small. 
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Suamarv 



This study tested whether the associative structure of individual 
students could be used to predict their performance in a particular course, 
Ti^enty representative terms in social psychology were presented to students 
in two sections of an Introductory course and their instructor, with the 
request that they assign assoclabllity scale values to all possible pairs 
of terms* The matrices were then factored by a new, nonparametric method 
which yields rank-ordered dimensions* 

The two main hypotheses, that course grade varies with number of 
dimensions, and that course grade varies with agreement with the instructor's 
structure, were both confirmed, but with correlations only in the *3 to *4 
range* Subsidiary hypotheses linking performance to size and direction of 
change in structure and to specificity of matching with the instructor’s 
structure all failed* It appeared that considerable interference came from 
end-of-scale effects in the assoclabllity judgments. 

Further study of this crea seems worth while, in view of the significant 
though weak findings* In future research, the assoclabllity judgment scale 
should be improved by prescribing the score distribution, larger amounts of 
data should be collected on fewer students so that a detailed analysis of the 
change process is possible, and further dependent variables, such as students' 
feelings about and degree of understanding of the instructor and of different 
books, should be added. 

Evidence was presented in an appendix of the usefulness of the nev7 
nonparametric factoring method, by means of comparisons with standard para- 
metric solutions* 
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Appendix A 

Mopsy : The Ilatrix Optimizing ^stern 



A useful factor analytic tool must reduce and abstract from the data 
in such a way that its basic, internal relationships are made clear. This 
is a complex requirement. Any factor analytic method, of course, boils 
down a matrix, but two problems then arise. One is that frequently a process 
of spinning out takes place, by which a large number of factors are generated. 
Beyond a certain number, these no longer clarify the data, but become in their 
turn a problem. Reports of factor analysis results commonly label one or more 
of the resultant factors "uninterpretable", or as due to error variance, or, 
worse, name factors with uninterpretable names or by listing their high values. 
Oddly enough, these difficulties seem to arise from the mathematical precision 
with which the factoring operation is defined, so that it is able to extract 
results below the level of error variation in the matrix. Unfortunately, it 
has been difficult to decide objectively when the factoring should stop. 

The second problem is related to the problem of interpretability. All 
the standard factor analytic methods put out sets of points located on 
arbitrary dimensions. In order even to attempt interpretation, the axes must 
be rotated, and the various available criteria for rotation, though clearly 
definable and repeatable, are simply different attempts to define mathematically 
a result that looks good on graph paper. My colleague, David Beards lee, who 
has had occasion to work extensively with factor analysis results, tells me 
that he has seen several sets of data for which the varimax and quartimax 
(the two most popular rotation methods) results were different, and that 
furthermore their results are sensitive to the starting point of the rotation. 
Indeed, some prograioners have introduced initial random rotations into their 
factor data in an effort to break up the effects of particular starting points. 
Rotation, then, appears to add a further imponderable to the already vexing 
problem of multiple factor interpretation. 

All of this is serious enough with input consisting of correlation co- 
efficients with a high degree of accuracy based on large samples. My data 
is on individuals; it is therefore fragile and filled with error. If I use 
an elephant gun on a mouse, I may not be able to find the mouse to tell 
whether I*ve hit it, VJhat seems to be called for instead is some sort of 
non-parametric factor analysis, which will not pretend that the data is more 
exact than it is. 

I was led to my proposed solution of this problem by consideration of 
the traveling salesman problem; how to visit all the points in a fully con- 
nected network and return home, all by the shortest route, (I understand 
that no rigorous solution exists for that problem,) I was not concerned 
with the return home, however, wishing rather to find the shortest path from 
one "end" of the set of points to the other. An association matrix may be 
regarded as isomorphic to a set of points with their closeness (rather than 
distance) represented at the intersections of the rows and columns. The 
matrix entries must all be positive, but a correlation matrix with negative 
numbers may easily be rescaled, Uhat we want is a reordering of the matrix 
so that the highly associated words are near each other in the order, with 
the result that the high values in the matrix are found near the diagonal 
and the low orus in the corners. In order to achieve this I treat the 
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association values in a row as weights along a rod and calculate the center 
of gravity i I then reorder the matrix by ranking the centers of gravity, 
repeating the whole process until the order stops changing* 

It*s not quite so simple as that. In order to prevent irreducible and 
meaningless oscillation, diagonal values must be set equal to the largest 
value in the row, and in order to prevent a certain class of false optimal 
solutions the matrix must be started as a 2 x 3 and gradually increased to 
n X n, starting with the rows having the highest sums and working down. 

The output of the program is a rank order that represents the pro- 
jections of a set of points onto a "best" axis drawn through them. The 
axis will tend to be drawn between extreme "corner" points in the space, 
but may not if the "tilt" of points nearer the center is sufficiently strong. 
What we seem to have, then, is Factor I, already rotated, l^hen applied to 
a word association matrix or a correlation matrix, the program turns out an 
order which is subjectively meaningful. 

Before describing the process of defining further dimensions, let me 
comment on the problem of rigor, I have been unable to define rigorously 
what I am doing. My colleague William Hammer le, of the Engineering School, 
was able to show that a rigorous ordering can be provided one allows it 
to be interval scaled, but that the proof may not hold for an ordinal scale. 
Unfortunately, the accompanying algorithm requires the solution of n-1 sets of 
equations, having each n-2 equations in as many unknowns (where n x n is the 
size of the matrix). Hammer le believes that it would be possible to reduce 
this requirement somewhat by further analysis, but the probable high cost 
in computer time and the necessity of accepting an interval scale output 
led me to decide in favor of the strictly pragmatic approach, which seems 
to have good empirical justification. Ham erle's approach did show me, 
nevertheless, that the ordering process should be limited to symmetric mat- 
rices, The program is designed to handle nonsymmetric input matrices by 
averaging the corresponding upper and lovfer triangle values. 

To find a second and further dimensions, we need a process for extracting 
from the matrix the results of the first ordering, remembering to make no 
assumptions about statistical distribution within the matrix. To do this, I 
find the actual frequency distribution and, as it were, deal it out, high 
values first, starting with the centermost off-diagonal (r^ C 2 , r 2 C 3 ,^a'» 
tn-i and working out. From each cell I subtract the lowest distribution 
value assigned to that off-diagonal, changing negatives to zero, (To keep 
the negatives is tantamount to assuming that the first order is an equal- 
interval scale, which, in general, it is not,) Then the process is repeated 
with the residuals. 

Empirical tests on sample distributions of points in 2- and 3-dimensional 
spaces showed good results both by eye and by correlations between input 
associations and output distances on the order of -,9 (the negative sign 
because association and distance are opposites). However, when data con- 
sisting of associative overlap scores or associability scale judgments was 
used as input, the interpretaoility remained high but the input-output 
correlation dropped - to ,6 for overlap scores and in some cases to ,3 for 
scale judgments. Worse yet, the stopping rule that took effect when the 
correlation ceased improving with added dimensions often cut off meaningful 
dimensions in the data. 

After a good deal of trial and error, I established a purely internal 
criterion, based on the tendency of the program to find successive orthogonal 
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dinensions. The argument is that as long as a new, independent ordering can 
be found there is still information in the matrix, but when orderings are 
like earlier ones, little new information is being added« To translate 
this into a practical stopping rule, I treated each new dimension coming out 
of the analysis as having a somewhat higher marginal cost. Factor analysis, 
after all, is supposed to simplify data* The cost function was defined as 
follows: when the sum of squares of rank order correlations between the last 
dimension and all previous ones, multiplied by the number of dimensions, is 
equal to or greater than 1, the cost of throwing away information is less 
than the cost of adding a dimension, and the last dimension is discarded. 

To give some feeling for the operation of this criterion, the following table 
shows, for each dimension number, the single correlation and the average 
correlation, either of which will, if equalled or exceeded, cause factoring 
to cease: 



# 


Single 


Average 


2 


.71 


.71 


3 


.58 


.42 


4 


.50 


.29 


5 


.45 


.23 


6 


.41 


.19 



Driver (1962) suggested that uneven weighting of dimensions indicates 
a low level of complexity. Since uneven weightings in Ilopsy cause repetition 
of information, Mopsy should cut off sooner on low- complexity matrices, 
where parametric analysis would find more factors. 

The Mopsy program is available on request at the Psychology Department, 
Oakland University, It is writte'^ in IBM Symbolic Programming System for 
the IBM 1620 with 40,000 decimal aigits of core storage, on line printer, 
and disk packs. On that apparatus, running time in minutes is approximately 
n^/100, where n is the matrix size. Complex data may run up to 257, longer. 

Since this method is not rigorously defined, it must depend on its own 
output for justification. The figures on the succeeding pages are Mopsy 
output for sets of data already subjected to parametric factor analysis by 
other investigators. 

Figure 1 shows the Mopsy solution for Thurstone's ( 47 ) Box Problem, 
Thurstone*s solution, as might be expected, comes up with three dimensions, 

X, Y, and Z, or length, width, and depth. Note that the Mopsy solution 
represents the relations among these measurements as a triangle in a plane, 

Thur stone finds it convenient to graph his results in a plane, too, and when 
he does the resulting diagram is almost identical with Figure 1. This 
points up the fact that Mopsy solutions tend to find the just-sufficient 
number of dimensions rather than a larger, absolutely "correct" number. 

Figure 2 gives the Mopsy solution for Harman's (' 16 ) physical measure- 
ments problem. Like the principal-factor solution, Mopsy finds two clusters o 
The existence of a general factor, which Harman calls "growth", is expressed 
by Mopsy as a correlation between the first and second dimensions. 

Figure 3 gives the Mopsy solution for Harman's ( 16 ) emotional traits 
problem. The principal-factor solution gives a "general emotionality" 
factor, which I find meaningless, and a second factor, almost identical with 
Mopsy* s factor I, which Harman calls "egocentrism", I would characterize 
the Mopsy solution as follows: Factor I: not-acted-upon vc, rxted upon; Vacter 

II: acting vs, not acting; Factor III (obviously) good vs, bad, Harman's "gcncj'a 



emotionality" could be described as the nearness to the center of the I'lopsy 
diagrams 

Figure 4 is the Mopsy solution for Harman* s ( IG ) political variables 
problem, based on the traits of voting districts* He finds two factors, 
"Traditional Democratic Voting" and "Home Permanency"* I would suggest 
labeling mine Social Class (I) and something like "Identification with the 
local community" (II)* It is hard to judge bea7een these - the preference 
would depend on what one could do conceptually with the two formulations* 

The OSS assessment variables (36)ai*c rcprcGontccl’ ?'.n Figure 5* The original 
parametric solution for that matrix produced four factors: Adjustment 

(emotional stability, social relations, security). Effective Intelligence 
(effective intelligence, observing and reporting, propaganda skills, over-all). 
Physical Energy (energy and Initiative, leadership, physical ability), and 
Authoritative Assertion (energy and initiative, leadership)* Motivation for 
assignment had no high weightings, but came closest to Adjustment* The 
Mopsy solution displays all those groups, but in two dimensions, apparently 
cognitive-noncognitive (I) and internal-external (II)* 

The examples displayed here demonstrate, I think, that Mopsy output 
makes sense, but that it is different from parametric factor analysis output. 
VJhlle it may be the case, as I have argued, that the factors come already 
rotated, as it were, the Mopsy diagram doesn*t look like "simple structure*" 
Because the rank ordering insists on uniform spacing in the distribution, 
the points spread out over the whole space rather than forming the typical 
tight clusters of parametric analysis* Personally, I find such a spacing 
easier to work with* What is most valuable, I feel, are the topological 
relationships among the points; absolute distances are much harder to inter- 
pret, and less reliable* 

In any case, I hope that other people will see fit to try out this 
method* Perhaps someone will be able to demonstrate rigorously why it v/orks 
and what its faults are* 
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